Papers with downstream models
EMCompress: Video-LLMs with Endomorphic Multimodal Compression (2026.findings-acl)
Copied to clipboard
| Challenge: | Static, sparse frame sampling either dilutes evidence across task-irrelevant segments at significant cost or misses fine-grained temporal semantics altogether. |
| Approach: | They propose a novel task that compresses multimodal input while preserving answer invariance across reasonable downstream models. |
| Outcome: | The proposed task surpasses prior methods by 0.40 F-1 with competitive query rewriting. |
A Pipeline for Generating, Annotating and Employing Synthetic Data for Real World Question Answering (2022.emnlp-demos)
Copied to clipboard
| Challenge: | Question Answering (QA) is a growing area of research . state-of-the-art QA models struggle on out-of domain documents without fine-tuning . |
| Approach: | They propose a pipeline for validating and training QA data and an interface for human annotation. |
| Outcome: | The proposed pipeline improves QA performance on domain-specific datasets while preserving the accuracy of the model. |
Learning to Prioritize: Precision-Driven Sentence Filtering for Long Text Summarization (2022.lrec-1)
Copied to clipboard
| Challenge: | Neural text summarization models are limited by their maximum input length, posing a challenge to summarizing longer texts comprehensively. |
| Approach: | They propose a pre-processing layer that removes low-quality sentences in articles to improve existing summarization models. |
| Outcome: | The proposed approach improves state-of-the-art summarization models on WikiHow and Reddit TIFU datasets by 3.84 and 8.57 points on the full test set and the long article subset. |
Prototype-Representations for Training Data Filtering in Weakly-Supervised Information Extraction (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Weak supervision and data programming are powerful tools to support information extraction models. |
| Approach: | They propose a prototype-based method to denoise weakly supervised training data . they use a model to model the correct contexts for a given target value . |
| Outcome: | The proposed method achieves 9% accuracy gain in attribute value extraction in e-commerce websites. |
Interpretable Entity Representations through Large-Scale Typing (2020.findings-emnlp)
Copied to clipboard
| Challenge: | In traditional methods for natural language processing, entities are embedded in dense vector spaces with pre-trained models. |
| Approach: | They propose an approach to creating entity representations that are human readable and achieve high performance on entity-related tasks out of the box. |
| Outcome: | The proposed representations are vectors whose values correspond to posterior probabilities over fine-grained entity types, indicating the confidence of a typing model’s decision that the entity belongs to the corresponding type. |
Pearl: A Review-driven Persona-Knowledge Grounded Conversational Recommendation Dataset (2024.findings-acl)
Copied to clipboard
Minjin Kim, Minju Kim, Hana Kim, Beong-woo Kwak, SeongKu Kang, Youngjae Yu, Jinyoung Yeo, Dongha Lee
| Challenge: | Existing datasets for conversational recommender systems lack specific user preferences and explanations for recommendations . current datasets lack specific preferences, hindering high-quality recommendations despite advances in large language models . |
| Approach: | They propose to synthesize a conversational recommendation dataset with persona- and knowledge-augmented LLM simulators to address these challenges. |
| Outcome: | The proposed dataset outperforms baselines in human and automatic evaluations. |
Bigger Data or Fairer Data? Augmenting BERT via Active Sampling for Educational Text Classification (2022.coling-1)
Copied to clipboard
| Challenge: | Pretrained Language Models (PLMs) encode bias against protected groups in the representations they learn, which may harm the prediction fairness of downstream models. |
| Approach: | They propose to quantify the awareness that a pretrained language model (BERT) has regarding people’s protected attributes and augment it to enhance prediction fairness of downstream models. |
| Outcome: | The proposed method improves fairness and accuracy of models by inhibiting the awareness of protected attributes in the PLMs. |
NUTMEG: Separating Signal From Noise in Annotator Disagreement (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent work suggests that annotators may have genuine disagreements, but few models separate signal from noise in annotator disagreement. |
| Approach: | They propose a Bayesian model that removes noisy annotations from training data while preserving systematic disagreements. |
| Outcome: | The proposed model outperforms models trained on NUTMEG-aggregated data. |
Attending via both Fine-tuning and Compressing (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that attention mechanisms can improve models' interpretation, but they are not explicable. |
| Approach: | They propose a framework consisting of a learner and a compressor to purify attention scores . they propose to fine-tune and compress the attention mechanism to obtain a more faithful explanation . |
| Outcome: | The proposed framework improves performance and interpretability on eight benchmark datasets. |
Deep Contextualized Word Representations (N18-1)
Copied to clipboard
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, Luke Zettlemoyer
| Challenge: | a new type of deep contextualized word representation is proposed for language understanding problems . word vectors are learned functions of the internal states of a deep bidirectional language model . |
| Approach: | They propose a new type of deep contextualized word representation that models complex features of word use and how they vary across linguistic contexts. |
| Outcome: | The proposed representations improve the state of the art across six challenging NLP problems. |
BPE-knockout: Pruning Pre-existing BPE Tokenisers with Backwards-compatible Morphological Semi-supervision (2024.naacl-long)
Copied to clipboard
| Challenge: | Subword tokenisation is a mainstay in natural language processing (NLP) it allows the representation of an infinite space of text with a finite set of units. |
| Approach: | They propose to use byte-pair encoding to represent an infinite space of text with a finite set of units by removing subwords from the BPE vocabulary without impeding further use of merges that relied on them. |
| Outcome: | The proposed method improves token-based tasks in English, Dutch and German while saving training costs. |
Rotate King to get Queen: Word Relationships as Orthogonal Transformations in Embedding Space (D19-1)
Copied to clipboard
| Challenge: | Word embeddings have a notable property that word relationships can exist as linear substructures in the embeddable space. |
| Approach: | They propose an alternative way in which downstream models might learn these relationships: orthogonal and linear transformations. |
| Outcome: | The proposed model can learn such relationships as geometric translations, but there is no evidence that it is exclusively accurate. |
Data Quality Issues in Multilingual Speech Datasets: The Need for Sociolinguistic Awareness and Proactive Language Planning (2025.acl-long)
Copied to clipboard
| Challenge: | Despite their growing importance, the quality of these datasets remains under-researched. |
| Approach: | They propose guidelines and recommendations to address quality issues in future dataset development . they find that macro-level issues are more prevalent in less institutionalized, often under-resourced languages . |
| Outcome: | The results highlight the need for proactive language planning and enhanced data quality control in the process of automatic speech recognition dataset creation. |
A Multi-dimensional study on Bias in Vision-Language models (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on the issue of bias in joint Vision-Language models . pre-trained models complete a neutral template with a hurtful word 5% of the time . |
| Approach: | They propose to use a multi-dimensional bias metric to investigate bias in English VL models . they use gender, ethnicity, and age as dimensions to analyze bias in VLs . |
| Outcome: | The proposed model is based on gender, ethnicity, and age as dimensions. |
Lawyers are Dishonest? Quantifying Representational Harms in Commonsense Knowledge Resources (2021.emnlp-main)
Copied to clipboard
| Challenge: | Commonsense knowledge bases are mostly human-generated and reflect societal biases . a filtering-based approach can reduce the issues in both resources and models but leads to a performance drop . |
| Approach: | They propose a filtering-based approach to mitigating representational harms in ConceptNet and GenericsKB . they propose filtered-based approaches can reduce issues in both resources and models but leads to performance drop . |
| Outcome: | The proposed approach reduces issues in resources and models but leads to performance drop . the paper proposes a filtering-based approach that reduces biases but leaves room for future work . |
RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputs (2023.acl-long)
Copied to clipboard
| Challenge: | Despite their success, even the largest language models make mistakes. |
| Approach: | They propose a framework where one language model can generate critiques to improve its peer's performance. |
| Outcome: | The proposed framework improves the performance of a fixed model 200 times its size by 10% over other models. |
Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods to debias word embeddings from human-generated corpora inherit strong gender bias . prior work has suggested removing gender component from pre-trained word embeds or compressing gender information into a few dimensions of the embeddable space . |
| Approach: | They propose a technique that purifies word embeddings against inferred gender subspaces . they propose to preserve distributional semantics of pre-trained word embeds while reducing gender bias . |
| Outcome: | The proposed technique preserves distributional semantics of pre-trained word embeddings while reducing gender bias to a larger degree than prior approaches. |
Fair NLP Models with Differentially Private Text Encoders (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Encoded text representations often capture sensitive attributes about individuals, raising privacy concerns and making models unfair to certain groups. |
| Approach: | They propose an approach that combines privacy and adversarial training to learn private representations which induces fairer models. |
| Outcome: | The proposed approach improves on four NLP datasets and shows that privacy and fairness can positively reinforce each other. |
“You are grounded!”: Latent Name Artifacts in Pre-trained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models perpetuate biases originating in their training corpus to downstream models. |
| Approach: | They focus on the representations of given names in pre-trained language models and show that name perturbation can have an effect on downstream tasks. |
| Outcome: | The proposed model can be used to model the representation of given names in pre-trained language models on reading comprehension probes where name perturbation changes the model answers. |
Tokenization is Sensitive to Language Variation (2025.findings-acl)
Copied to clipboard
| Challenge: | Variation in language is often linked to regional, social, and contextual factors. |
| Approach: | They propose a method to estimate tokenizer impact on downstream LLM performance . they pre-train BERT models with the popular Byte-Pair Encoding algorithm . |
| Outcome: | The proposed model improves on Rényi efficiency and other metrics on language variation. |
Plug-and-Play Knowledge Injection for Pre-trained Language Models (2023.acl-long)
Copied to clipboard
Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Huadong Wang, Deming Ye, Chaojun Xiao, Xu Han, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou
| Challenge: | Existing knowledge injection methods are not suitable for enhancing pre-trained language models with external knowledge bases. |
| Approach: | They propose a plug-and-play knowledge injection method where knowledge bases are injected into frozen existing downstream models by a knowledge plugin. |
| Outcome: | The proposed method improves the performance of knowledge injection on knowledge-driven tasks while keeping model parameters frozen. |
Pre-trained Speech Processing Models Contain Human-Like Biases that Propagate to Speech Emotion Recognition (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work has established that a person’s demographics and speech style affect how well speech processing models perform for them. |
| Approach: | They propose a method to detect bias in pre-trained models by using word embedding association tests in natural language processing to quantify bias in models' representations of different concepts. |
| Outcome: | The proposed method detects bias in pre-trained models and can have real-world effects. |
Effects of diversity incentives on sample diversity and downstream model performance in LLM-based text augmentation (2024.acl-long)
Copied to clipboard
| Challenge: | generative large language models (LLMs) have found their application in data augmentation tasks, where small numbers of text samples are paraphrased and then used to fine-tune downstream models. |
| Approach: | They propose to use taboo words, hints by previous outlier solutions, and chaining on previous outliest solutions to augment text datasets as part of instructions to LLMs augmenting text dataset. |
| Outcome: | The proposed methods increase diversity of generated texts, but performance is highest with hints. |
FIPO: Free-form Instruction-oriented Prompt Optimization with Preference Dataset and Modular Fine-tuning Schema (2025.coling-main)
Copied to clipboard
| Challenge: | naive prompts can enhance the task performance of large language models, but they are resource-intensive. |
| Approach: | They propose an automatic prompt optimization method that refines naive prompts according to task outputs from in-box testing models. |
| Outcome: | The proposed method is based on a large-scale dataset and performed fairly across multiple models. |
VOYAGER: A Training Free Approach for Generating Diverse Datasets using LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are used to generate synthetic datasets but lack diversity . prior work has noted that such generated data lacks diversity - a problem that requires domain expertise. |
| Approach: | They propose a principled approach that optimizes a mathematical quantity that optimize the diversity of the dataset using determinantal point processes. |
| Outcome: | The proposed method improves diversity by 1.5-3 times compared to baseline approaches. |
Personality Vector: Modulating Personality of Large Language Models by Model Merging (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to induce personality in large language models (LLMs) fail to capture the continuous nature of human traits. |
| Approach: | They propose a method for personality modulation in large language models by model merging by subtracting weights of pre-trained models from those of fine-tuned models. |
| Outcome: | The proposed method allows LLMs to exhibit desired personality traits without additional training. |
Rolling Out Data Quality Overnight, without losing the plot: A Multi-Agent System for Speech Data Quality Management (2026.findings-acl)
Copied to clipboard
Rishabh Kumar, Abhinav Painuli, Chriss Philip Saji, Devesh Soni, Amrith Krishna, Ganesh Ramakrishnan
| Challenge: | Using automation to improve quality management is expensive and resource-intensive for speech datasets. |
| Approach: | They propose a natural language-driven agentic framework that compiles user requirements into dependency-aware DAG workflows over modular tools for audio, transcript, and metadata verification. |
| Outcome: | The proposed framework achieves 80-90% agreement with expert verification while requiring less than 20% of the cost and time of manual QC. |